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We present an observation of B — > rj K* . The data sample corresponds to 232 million BB 
pairs collected with the BABAR detector at the PEP-II asymmetric-energy B Factory at SLAC. 
We measure the branching fractions (in units of 10 -6 ) B(B° — > rj'K* ) — 3.8 ± 1.1 ± 0.5 and 
B{B + -> r/'K* + ) = 4.9±i;? ± 0.8, where the first error is statistical and the second systematic. A 
simultaneous fit results in the observation of B — » rf K* with B(B — » rf K*) = 4.llJ'g ±0.5. We also 
search for _B — > rj'p and 77' /o(980)(/o — > 7r + 7r _ ) with results and 90% confidence level upper limits 
B(jB + _» v > p+) = 8.7±i:5±?:i « 14), B(B° -> r?V°) < 3.7, and 6(5° -» 7?7o(980)(/ - < 
1.5. Charge asymmetries in the channels with significant yields are consistent with zero. 

PACS numbers: 13.25.Hw, 11.30.Er 



Decays of B mesons involving the flavor-changing neu- 
tral current transition b — > s are an important place 
to search for evidence of physics beyond the Standard 
Model. A comparison of the amplitude sin 2/3 of time- 
dependent CP violation in the neutral CP eigenstates 
J/i/jK® and rfK® provides one of the most sensitive 
tests [l|. In order to unambiguously interpret the time- 
dependent CP violation measurement in rj'K® it is impor- 
tant to understand the full set of underlying amplitudes 
by making measurements of branching fractions in the 
rfK* decays. 

In B decays to final states comprising rf'^K^*' the final 
states rfK* and r/K are suppressed, and the final states 
rf K and rjK* are enhanced. Two explanations of the ex- 
perimentally observed pattern differ substantially in the 
details of the suppression for B — > rfK* 0, 0|. From 
previous experimental data and flavor SU(3) arguments 
it is expected that the branching fractions for B — > rfK* 
are less than 10~ 5 [4j. The related decays B — > rf p oc- 
cur via CKM suppressed tree diagrams and are expected 
to be small. Theoretical approaches using QCD factor- 
ization and perturbative QCD Q predict branching 
fractions for B + — ► r)'p + of 6-9 x 10~ 6 and for B° — > rj 'p° 
of 0.5-2 x lfr 7 . 

In this Letter, we present searches for B — > rj'K*, 
B -> rfp and B° -> r//o(980)(/ -> tt+tt"), which shares 
the same final state as B° — > 77' p Q . Throughout this Let- 
ter, charge conjugation is implied. Results are obtained 
from unbinncd, extended maximum likelihood (ML) fits 
to data collected with the BABAR detector at the PEP-II 
asymmetric e + e _ collider located at the Stanford Linear 
Accelerator Center. The BABAR detector and relevant de- 
tails specific to this analysis are described elsewhere 
The analysis uses 211 fb _1 of data recorded at the T(4S) 
resonance, corresponding to 232 million BB pairs, and 
closely follows the approach described in Ref. [8(. 

We select 77', K*, p, 77, K® and tt° candidates through 
the decays 77' -> 77^+71-" (jt^J, rf -> p°~f (77^), 
K*° -> K+TT-, K*+ K° s ir+ (K*+ 0t+ ), K*+ -> K+n° 
(K*+ + ^ ), p°_ (and /o) - vr+vr-, p+ % ^+vr°, 77 77, 
i£g — > 7r + 7r _ and 7r° — > 77. We impose the following 
requirements on candidate invariant masses, in MeV/c 2 : 
910 < (™ ??7 r7r,"ip 7 ) < 1000 for 77', 755 < < 1035 

for the K*, 510 < m W7r o < 1070 for p+ and 510 < 
< 1060 for p° (/ ), 490 < m 77 < 600 for 77, 



486 < < 510 for K° s and 120 < m 77 < 150 for 

7T°. For the masses of the 77', K* and p, which will be in- 
cluded as observables in the ML fit described below, the 
selection is wide enough to allow for a parameterization 
of the background. For K® candidates we require a flight 
distance of at least three times its estimated uncertainty. 

We also use the helicity- frame decay angle Oh of K* , p, 
and /o(980). The helicity frame is defined as the vector 
meson rest frame with polar axis along the direction of 
the boost from the B rest frame. The angle Oh is the an- 
gle between the polar axis and the flight direction of the 
charged resonance daughter. For K*° and p° the kaon 
candidate and the positively charged pion, respectively, 
are used to define that angle. We use mode dependent se- 
lection criteria on cosQh, with the lower bound between 
—0.95 and —0.70 and the upper bound of either 0.95 or 
1.00. Decay modes suffering from higher combinatoric 
background due to low momentum pions have the tighter 
cuts applied. The helicity has a cos 2 9h distribution for 
K* and p signal events and is flat for the /o(980). 

All charged pion candidates are required to have par- 
ticle identification (PID) consistent with pions and in- 
consistent with protons, kaons, and electrons. No such 
requirement is made of K~ daughters. Charged kaon can- 
didates are required to have PID consistent with kaons 
and inconsistent with pions, protons and electrons. 

We form B meson candidates by combining an 77' can- 
didate with either a K* or p candidate. B meson candi- 
dates are characterized kinematically by the energy sub- 
stituted mass, tties = (s/4 — p%) x ^ 2 and the energy dif- 
ference AE = Eb — \fsj1 where (Eb,Pb) is the four- 
momentum of the B candidate, expressed in the T(4S) 
frame and y/s is the e + e~ center of mass energy. Sig- 
nal events peak at zero for AE and at the B mass for 
rriES, with typical resolutions of 20 MeV and 3.0 MeV/c 2 , 
respectively. We require 5.25 < tties < 5.29 GeV/c 2 
for all modes, -0.2 < AE < 0.150 GeV for modes 
where the vector meson decay includes a neutral pion 
and -0.2 < AE < 0.125 GeV otherwise. 

Backgrounds arise primarily from random combina- 
tions of particles in continuum e + e~ — ► qq (q = u, d, s, c) 
events. To reject these events, we employ the angle 0t 
in the T(4S I ) frame between the thrust axis of the B 
candidate's daughters and that of the remaining parti- 
cles in the event. Continuum events are produced well 
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above threshold, with a jet-like topology resulting in a 
distribution of | cos6*t| that is sharply peaked near 1 for 
candidates formed in such events. Events containing true 
T(4S) decays are produced near threshold with particles 
distributed isotropically, resulting in a uniform distribu- 
tion of | cos#r|- We require | cos#r| < 0.9 for decays with 
r]L n7r , and | cos#t| < 0.75 for the higher-background r( 
decays. Due to large backgrounds in r)' , we only use the 
r;' ;7r7r decay in reconstructing B — + f/p// (980). 

Additional discrimination against continuum back- 
ground occurs in the ML fit and is provided by a Fisher 
discriminant, J- . This is a linear combination of discrim- 
inating variables with weights chosen to maximize the 
separation between signal and continuum background. 
T contains the angles of the B momentum and B thrust 
axis with respect to the beam axis, the i?-flavor tagging 
category [§], and the zeroth and second angular moment 
of the energy flow in the rest of the event with respect to 
the B candidate thrust axis [§]. 

After selection, events containing multiple B candi- 
dates occur less than 30% of the time. In such cases, we 
choose the B candidate with the rj' mass closest to the 
Particle Data Group (PDG) value [HJ. 

We use Monte Carlo (MC) simulation [ll| for an ini- 
tial survey of background from BB events and to identify 
for detailed study any decays that are not rejected by 
candidate selection. The remaining background is com- 
posed almost entirely of charmless resonant B decays, 
especially B — > rj'K. We account for B backgrounds by 
including in the ML fit an additional component which 
models these charmless, resonant decays. Backgrounds 
arising from charmed B decays have been studied and 
found to be negligible or accounted for by our continuum 
background model. Backgrounds from non-resonant B 
decays have been found to be consistent with zero. 

We determine yields and charge asymmetries (A c h = 
(n + — n~)/(n + + n~)) for each decay chain from a ML 
fit with the observables AE, ihes, J 7 , m n ', the mass of 
the candidate vector meson my, and 7i = cos 9h- For 
charged (neutral) B decays, is defined as the number 
of B^ decays (final states with K^). For each event i 
and hypothesis j (signal, continuum, BB), we define the 
probability density function (PDF) as a simple product 
of the individual observable PDFs: 

For the t]'tt + tt~ final state, a fourth hypothesis is added 
to account explicitly for a possible tj' f signal. 
The total likelihood function is then given by 

* 3 

where N is the number of events in the sample and rij is 
the yield of events of hypothesis j to be found by max- 
imizing C. In addition to the yields and A c h for each 



hypothesis, parameters describing the continuum PDFs 
are also allowed to vary (see below). 

We parameterize the PDFs for peaking observables 
with either a single or asymmetric Gaussian, sum of two 
Gaussians, or a Breit-Wigner as required. Slowly vary- 
ing observables are described by low degree polynomials 
or phase-space motivated functions Q. Several PDFs 
require linear combinations of peaking and non-peaking 
shapes. We parameterize the /o(980) mass and width 
using measured values [l2j ]. 

For the signal and BB background components we 
determine the PDF parameters from simulation. Con- 
trol samples with topologies similar to our signal (e.g., 
B~ — ► D°tt~) are used to verify and adjust simulated 
resolutions 8]. For the continuum background we ob- 
tain initial PDF parameters from data excluding the AE 
and tties signal region (sideband). We further refine the 
continuum PDFs by letting as many parameters as fea- 
sible vary in the fit to the full data. The final fitted 
continuum background PDF parameters are found to be 
in close agreement with their initial values. 

We apply several tests to the fitting procedure for vali- 
dation before implementing it on the data. In particular, 
we evaluate any possible bias in our event yields due to 
our neglect of small correlations between the observables, 
which our PDFs ignore by construction. We determine 
the bias by fitting ensembles of simulated continuum ex- 
periments generated from the PDF into which we em- 
bed the expected number of signal and BB background 
events randomly taken from samples of fully simulated 
MC events. Measured correlations in the sideband data 
(pure qq) are found to be small. The measured biases for 
each decay chain are given in Table U 

We compute the branching fraction for each decay by 
subtracting the fit bias from the measured yield and di- 
viding the result by the efficiency (determined from sim- 
ulation and ancillary studies), the product of the daugh- 
ter branching fractions, and the number of produced BB 
pairs. We assume equal decay rates of the T(45) to 
B + B~ and B°B°. In Table Q] we show for each decay 
the measured branching fraction, event yield, efficiency 
and daughter branching fraction as well as A c h- 

Measurements for separate decay chains are combined 
by adding the values of — 21n£ as functions of branch- 
ing fraction, taking appropriate account of correlated and 
uncorrelated systematic uncertainties (described below) 
8]. The significance is taken as the square root of the 
difference between the value of — 21n£ (including sys- 
tematics) for zero signal and the minimum. For modes 
where the combined significance is less than 4 standard 
deviations, we quote 90% confidence level (C.L.) upper 
limits. We compute these as the branching fraction be- 
low which lies 90% of the total likelihood integral in the 
positive branching fraction domain. 

For modes with evidence of a signal, we show in Fig. [1] 
projections onto /ties and AE of subsamples (containing 
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TABLE I: Summary of results showing (from left): fitted signal yield n before bias correction, fit bias, detection efficiency 
e, product daughter branching fraction \\Bi significance S (including systematic uncertainties) in standard deviations, 
measured branching fraction B and signal charge asymmetry Ach for each mode. The values in parentheses are 90% C.L. upper 
limits. The result for B° -> 7?7q(980)(/ — > 7r + 7r ) includes the branching fraction for fo — > tv + tv , which is not well known. 
Results in bold face represent combined fits to multiple decay chains (when present). 



Mode 


tl (ev.) 


Bias (ev.) 


e{%) 




S(a) 


£(10~ 6 ) 




B -> r/K* 










5.6 


4.1 + n° ± 0.5 

— 0.9 




B° -> rj'K* 










4.3 


o o _1_ -i i ±nc 

*J«0 1 J_ . .L 1 \J > t> 


—0 OS + 9^ + 09 


ri K*° 


99 fi +7T 


+ 1.7±0.9 


19.0±f .2 


11.6 


3.9 


4 1+ 1 - 5 

^■ L -1.3 




ri K*° 


« 1+ 14 - 2 

— 12.7 


_|_Q K-L/l C 


16.9±1.1 


1Q 7 


9 n 


q q+1.9 
°-°-1.6 




B+ -> riK*+ 










3.6 


4.9ti;7 ± °- 8 (< 7 - 9 ) 


0.30±g:|? ± 0.02 


ri K* + 


f f 2 +5 - 7 

4.8 


+0.8±0.5 


18.0±1.2 


4.0 


3.2 






ri K* + 




+2.9±1.5 


15.8±1.1 


6.8 


1.2 






ri K* + 


r 0+5.4 
J - z -3.6 


+1.0±0.5 


f0.7±0.6 


5.8 


1.2 


z - y -2.6 




ri K* + 


o 1+12.1 
°' i -9.6 


-2.3±1.3 


8.0±0.5 


9.8 


0.5 


2 9+ 6 ' 7 




B° -+ rip 


14.9i^° 4 6 


+11.2±5.7 


22.8±1.4 


17.5 


0.3 


0.4±J™(< 3.7) 




B° -» 777oHir+7r-) 




-3.8±2.0 


25.4±1.6 


17.5 


0.2 


o.it : 6 4±°o-i« 1-5) 




B+ -+ V 'p+ 


O + 16.0 

<-* ' -° — 14.7 


+11.5±5.8 


13.0±1.0 


17.5 


3.2 


8.7±|:S±?:i« 14) 


-0.04 ±0.28 ±0.02 



63 - 85% of all signal events) enriched by a requirement 
on the ratio of the signal likelihood to the total likeli- 
hood. The likelihood is computed excluding the plot- 
ted variable. Figure [2] shows background-subtracted dis- 
tributions of the K*° mass and helicity obtained with 
the sPlot technique described in These plots illus- 
trate that the K n signal we observe is consistent with 
the i^*(892) and is polarized as one would expect in a 
pseudoscalar- vector B decay. 

Systematic uncertainties in this analysis are dominated 
by our knowledge of signal and BB background PDF 
modeling, along with the fit bias and the efficiencies of the 
track and neutral particle selections. Uncertainty due to 
continuum PDF modeling is largely incorporated into the 
statistical uncertainty since most continuum background 
parameters are allowed to vary in the fit. Uncertainties in 
the signal PDF parameters are estimated from compar- 
isons between data and MC in control samples. Varying 
the signal PDF parameters within these errors results in 
a mode dependent variation in signal yield of between 0.1 
and 1.6 events. 

The uncertainty in the fit bias is taken to be half of 
the correction. We estimate the uncertainty from BB 
modeling by taking half of the difference between the 
signal yield fitted with and without the BB component 
(0.2 to 10 events). The uncertainty due to non-resonant 
BB background is estimated by taking half the difference 
between the signal yield in the nominal fit and in a fit 
in which a non-resonant background component has been 
added (0.7 to 4.8 events). Uncertainties in reconstruction 
efficiency are determined from supplementary studies of 




m^GeV/c 2 ) AE(GeV) 
FIG. 1: (Color online) B candidate tues (left) and A.E 
(right) projections obtained with a cut on the likelihood 
(see text) for B° -> riK*°(top), B+ -> ri K* + (middle) and 
B + — > ri p + (bottom) . Submodes have been combined. The 
data are represented by points with uncertainties, full fit func- 
tions by solid curves, BB background by dashed, continuum 
by dotted and signal by dot-dashed curves. Depending on the 
decay, the plots contain 63 - 85% of all signal events. 



control samples. These include 0.8% per charged track 
(excluding daughters of the K®), 1.5% per photon, and 
1.9% for a K®. The systematic uncertainty in the num- 
ber of BB pairs is 1.1% [3. Published data provide 
the uncertainties in the £>-daughter product branching 
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m KV (GeW) 

FIG. 2: (Color online) Distributions of the Kn mass (Ze/i) and 
helicity (right) for the decay B° — > T]'K*°. Points with error 
bars: data, background subtracted with the sPlot technique, 
solid curve: signal PDF. 



fractions (3.4%). Uncertainties in the event selection ef- 
ficiency are 0.5-3% for the requirement on cosOt- 

We assign a systematic uncertainty on A c h of 0.02, 
based on studies of inclusive samples of kaons and B 
decays. This is due primarily to asymmetries in charged 
kaon identification and slow pion reconstruction. 

We present measurements for the decays B + '° — > 
T]'K* + ' Q and B + — > i]'p + . They allow the level of sup- 
pression of these decays, with respect to the enhanced 
rf K and r)K*, to be determined. A simultaneous fit of 
all charged and neutral rj'K* submodes results in the ob- 
servation of B — > rj'K* with a total significance of 5.6er, 
including systematics, as shown in Table HI The mea- 
surements place constraints on possible enhanced flavor- 
singlet contributions to these decays @, EH • These results 
are consistent with previous upper limits, where they ex- 
isted. In all cases, predictions based on SU(3) flavor sym- 
metry 0], QCD factorization [|| and perturbative QCD 
Q are in excellent agreement with our measured cen- 
tral values. Values of A c h are consistent with zero in all 
channels. 
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